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•  MSBL  supports  EN,  MP,  CM  Schools  and  the  Maneuver 
Support  and  Protection  Warfighting  Function  (WfF). 

•  MSBL  manages  a  virtual  Base  Defense  Operation 
Center  (BDOC). 

•  Through  One  Semi-Automated  Forces  (OneSAF)  and 
additional  sensor  software,  the  BDOC  can  simulate 
capabilities  of  future  Army  systems  under  development. 
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•  A  future  sensor  capability  would  allow  for  the  sharing  of 
information  between  sensors  and  systems  in  a  dynamic 
tactical  environment. 

•  The  BDOC  represented  a  virtual  base  camp,  which 
allowed  for  testing  of  the  representative  future 
capabilities. 

•  Problem:  How  does  this  future  sensor  capability  impact 
the  BDOC  Commander’s  mission  effectiveness? 
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•  Participants:  Six  Active  Duty  Soldiers  (3  SSG,  1  SFC,  2 
CPT)  served  as  BDOC  commanders  for  the  two  week 
experiment. 

•  Baseline  (current  sensor  arrangement  /  no  additional 
capabilities)  vs.  enhanced/  future  capability. 

•  The  baseline  scenario  made  operator  wait  approximately 
ten  minutes  to  regain  sensor  feed.  The  enhanced 
capability  run  allowed  for  immediate  sensor  availability. 
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Available  Experiment  Sensors 
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Boomerang 
Gunshot  Detector 


U 


Pelco  Camera 


Rapid  Aerostat  Initial 
Deployment  (RAID) 


Long  Range  Thermal 
Imaging  (LRTI) 


Cerberus 


Patrol  video  camera 


*  1  > 
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Unmanned  Aerial  System  (UAS) 


Initially  on  and  available  at  the  beginning 

Initially  off  but  available  during  the  scenario 

All  images  found  from  online  search. 
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OneSAF  Images 


Unmanned 
Aerial  System 
(UAS)  operator 


Radio  operator 
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•  Three  scenarios  (25  minutes  each): 

1 .  Three  vehicle  patrol  is  patrolling  outside  of  the  base 
camp.  At  five  minute  mark,  RAID  sensor  feed  is 
removed.  At  ten  minute  mark,  enemy  sniper  engages 
patrol. 

2.  No  enemy  action,  but  BDOC  loses  all  video  feed. 

3.  Civilian  protest  at  front  gate.  At  five  minute  mark, 
RAID  sensor  feed  is  removed.  At  ten  minute  mark, 
Vehicle  Borne  Improvised  Explosive  Device  (VBIED) 
explodes  outside  of  base  camp. 

•  Data  recorder:  recorded  time  to  detect,  identify,  decide, 
SALT  report,  radio  time,  survey  afterwards. 
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Constraints,  Limitations,  Assumptions 


•  Constraints: 

-  The  virtual  BDOC  environment  tested  the  participants’ 
use  of  this  future  capability.  No  real  world  testing  of 
actual  capabilities  was  available. 

•  Limitations: 

-  The  limited  experiment  preparation  time  did  not  allow 
for  a  full  spectrum  of  stakeholder  or  Warfighter  input. 

•  Assumptions: 

-  The  simulation  served  as  a  realistic  replication  of  a 
small  (50-299  Soldiers)  base  camp. 

-  All  of  the  participants’  input  is  equally  weighted. 

-  All  collected  will  be  used  for  the  experiment;  no 
outliers  will  be  discarded. 
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Multi-Attribute  Decision  Making  (MADM) 


Four  common  characteristics: 

1 .  There  are  multiple  objectives  and  attributes. 

2.  Objectives  &  attributes  conflict  with  each  other  to  some 
extent  and  tradeoffs  apply  (i.e.,  increasing  the  level  of 
achievement  for  one  objective  or  attribute  may  result  in 
a  decrease  in  the  level  of  achievement  for  one  or  more 
of  the  remaining  objectives  and  attribute). 

3.  Units  of  measurement  are  not  the  same  across  all 
attributes. 

4.  Purpose  of  analysis  is  to  evaluate  all  alternatives 
and/or  select  best  alternative. 
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Multi-Attribute  Decision  Making  (MADM) 


<*> 


1 .  Identify  stakeholders. 

-  Customer,  MSCoE  Leadership,  Subject  Matter  Experts 

2.  Identify  fundamental  objectives. 

-  Determine  impact  of  future  capability  to  BDOC 
Commander’s  threat  response. 

3.  Develop  the  qualitative  model. 

-  Understand  what  attributes  are  important  to  the 
stakeholder. 

4.  Develop  quantitative  value  model. 

-  Leads  to  Data  Collection  Management  Plan  (DCMP). 

-  Develop  and  Assess  value  functions  for  each  attribute. 
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13 


Qualitative  Model 


Baseline:  No  capability 
Enhanced:  Capability  on 


How  does  this  future  sensor  capability  impact  the 
BDOC  Commander’s  mission  effectiveness? 


Impact  to  threat 
response 
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33% 


Impact  to  user  control 


Situational 
Awareness 
(SA) 
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Time  to  respond 
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Quantitative  Model  /  Data  Collection  Plan 


U.S.ARHY 


s> 


Issue  and  (weighting) 

Essential  Elements  of  Analysis 
(EEA)  and  weighting 

Measure  of  Performance  (MOP) 
and  weighting 

Method  and 
(Units) 

Issue  1 .  (67 %):  How  does  the 
future  sensor  capability  impact 
threat  response? 

EEA  1.1  (67%):  How  does  the 
future  sensor  capability  impact 
situational  awareness  (SA)? 

MOP  1.1.1  (10%):  What  is  the 
time  from  incident  to  detection? 

Timer 

(seconds) 

MOP  1.1.2(10%):  What  is  the 
time  from  detection  to 

identification? 

Timer 

(seconds) 

MOP  1.1.3(10%):  What  is  the 
time  from  identification  to 

decision? 

Timer 

(seconds) 

MOP  1.1.4  (35%):  What  is  the 
time  from  incident  to  complete  a 
Size,  Activity,  Location,  Time 
(SALT)  report? 

Timer 

(seconds) 

MOP  1.1.5  (35 %):  After  a  sensor 
failure,  how  was  your  ability  to 
regain  situational  awareness 
impacted? 

Survey 

(Normalized  0- 
100  Scale) 

EEA  1 .2  (33%):  How  does  the 
future  sensor  capability  impact 
quantity  and  nature  of 
communications? 

MOP  1.2.1  (55%):  What  is 
percent  of  radio  communication 
time? 

Timer  (percent) 

MOP  1 .2.2  (45%):  How  many 
errors  are  in  the  SALT  report? 

Data  Recorder 
(observation) 

Issue  2.  (33%):  How  does  the 
future  sensor  capability  impact 
user  control? 

EEA  2.1(68%):  How  does  the 
sensor  data  feed  impact  user 
performance? 

MOP  2.1.2  (100%):  What  is  the 
workload  impact? 

Survey 

(Normalized  0- 
100  Scale) 

EEA  2.2  (32%):  What  are  the 
impacts  on  user  confidence? 

MOP  2.2.1  (100%):  How  does 
the  future  sensor  capability  impact 
user  confidence  in  responding  to 
a  threat? 

Survey 

(Normalized  0- 
100  Scale) 

Example  Attribute  Curves 
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Attribute  curves  determine  value  of  a  score  of  an  attribute, 
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Survey  Question 


How  did  the  future  sensor  capability  impact 
your  confidence  in  responding  to  a  threat? 


Greatly 

Diminished 

No  Change 

Enabled  (75) 

Greatly  Enabled 
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(50) 

(100) 

Score 
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Experiment  Data  Collection  Example 
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Data  from  one  participant  in  one  run 


Baseline 
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How  does  the  future  capability  impact  threat  response? 
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MOP  1.1.2(10%) 

to  identification? 

tir 
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Experiment  Data  Collection  Example 
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Data  from  one  participant  in  one  run 


Baseline 


Enhanced 


Value  from 

model 

Value 

input 

Value  from 

model 

Value  Input 

Issue  2. 

(33%) 

How  does  the  future  capability  impact  user  control? 

0.33 

Raw  Score 

Raw  Score 

EEA  2.1 

(68%) 

How  does  sensor  data  impact  user  performance? 

0.68 

MOP  2.1.2  (100%)  What  is  the  workload  impact? 

survey 

1 

50 

50 

11.22 

SO 

SO 
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0.32 

1 

50 

50 

5. 28 

EEA  2.2 

(32%) 

What  are  the  impacts  on  user  confidence? 

MOP  2.2.1(100%) 

How  does  the  future  capability 
impact  user  confidence  in 
responding  to  a  threat? 

61.5263 1  _  |  35.9436945 1 

f  39.69%| 

Percent  Overall  Utility  Increase 


25  min  run 

12:41  mins  on  radio  7:57 

mins  on  radio 

761.00 1  1500 1  50.73% 

477 

1500 1  31.80% 

seconds 

seconds 

37.32% 

Percent  radic 

)  time  reduction 
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•  60.71%  Overall  utility  Increase  with  the  future  sensor 
capability  enabled. 

-  Sensitivity  Analysis  showed  that  attributes  are  not  sensitive  to 
significant  Issue  and  EEA  weighting  changes. 


Participants’  Overall  Utility  Increase 


Each  attribute  can  be  individually  analyzed. 

-  Overall  time  spent  on  the  radio  decreased  by  27.5%. 


Participants’  Overall  Radio  Time  Decrease 


Summary 


•  Through  the  virtual  BDOC,  MSBL  can  accommodate 
contingency  base  and  protection  focused  experiments 
and  studies. 

•  MADM  is  a  straightforward  and  effective  method  of 
determining  “goodness”  of  a  system  or  capability  under 
evaluation. 

•  MADM  showed  a  60.7%  increase  in  utility  of  the  future 
sensor  capability,  compared  to  a  current  baseline. 
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Backup 
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Weight  of  Issue  1  (How  Does  Future  Capability  Impact  Threat 
Response?) 


Overall  utility  Increase  Impact  by  Varying  Issues  1  and  2  Weights 


Issue  and  (weighting) 


Issue  1 .  (67%):  How  does 
future  capability  impact 
threat  response? 


Essential  Elements  of  Analysis  (EEA)  and 
weighting 

EEA  1.1  (67%):  How  does  future  capability 
impact  situational  awareness  (SA)? 

EEA  1 .2  (33%):  How  does  future  capability 
impact  quantity  and  nature  of  communications? 


Issue  2.  (33%):  How  does 
future  capability  impact  user 
control? 


EEA  2.1  (68%):  How  does  sensor  data  impact 
user  performance? 


EEA  2.2  (32%):  What  are  the  impacts  on  user 
confidence? 


Overall  Utility  Increase  Overall  Utility  Increase 


Weight  of  EEA  1.1  (Keeping  Issue  Weights  Constant) 


Weight  of  EEA  2.1  (Keeping  Issue  Weights  Constant) 


Value 


Backup  -  Utility  Curve 
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Time  (s) 


Time  from  detection  to  identification 


Value 


Backup  -  Utility  Curve 


1 


Time  (s) 


Time  from  identification  to  decision 


Value 


Backup  -  Utility  Curve 


1 


Time  to  complete  a  SALT  report  -  from  incident 
to  completion? 


Percent  of  Time  using  the  radio  per  run 


Percent  of  phone/radio  communication  time 


Backup  -  Survey  Questions 


Survey  Question  1 :  Suppose  that  your  situational  awareness  is  100%  at  the  beginning  of  a  scenario. 
Indicate  the  value  of  how  you  regained  situational  awareness  once  the  sensor  failed. 


Greatly 

Diminished 

No  Change 

Enabled 

Greatly 

Diminished  (0) 

(50) 

(100) 

(150) 

Enabled  (200) 

Survey  Question  2:  How  did  you  feel  that  the  enhanced  system  impacted  your  ability  to  take  on  more 
work  load?  Choose  any  number  between  0  (greatly  diminished)  and  100  (greatly  enabled).  In  this 
question,  a  value  of  50  represents  no  change  between  the  baseline  and  enhanced  environments. 
Considering  the  first  run  (baseline)  as  a  base  line  of  50,  this  question  addresses  the  impact  that  the 


future  capability  had  on  the  participa 

nt’s  ability  to  take  on  more  work  load,  if  it  was  required, 

Greatly 

Diminished  (0) 

Diminished 

J25) 

No  Change 
J50) 

Enabled 

£5) 

Greatly 

Enabled  (100) 

Survey  Question  3:  How  did  an  enhanced  environment  impact  your  confidence  in  responding  to  a 


Greatly 

Diminished 

No  Change 

Enabled  (75) 

Greatly 

Diminished  (0) 

(25) 

(50) 

Enabled  (100) 

Survey  Question  4:  How  did  your  tactics,  techniques,  and  procedures  (TTPs)  change  in  the  BDOC 
with  a  baseline  vs.  ehanced  environment? 


Survey  Question  5:  What  risks  to  base  camps  operations  does  non  -  baseline  vs.  enhanced  uncover? 
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